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ABSTRACT 

We  present  here  a  new  scheme  to 
construct  a  neural  network  architecture 
based  on  the  physiological  properties 
biological  neuron  for  enhancing  its 
performance.  The  new  scheme  divides  every 
hidden  layer  into  two  parts  to  facilitate  the 
processing  of  0  and  1  separately  and  reduces 
the  total  number  of  interconnections 
considerably.  The  first  part  consist  of  units 
that  receives  signals  only  from  ‘1  state’  units 
of  the  immediately  lower  layer  and  are 
responsible  for  producing  excitation  units  in 
the  output  layer  i.e.  the  ‘1  state’  and  the 
second  part  consist  of  units  that  also  receives 
signals  only  ‘1  state’  units  from  the 
immediately  lower  layer  are  responsible  for 
producing  inhibition  of  the  units  in  the 
output  layer  i.e.  the  ‘0  states’. 

The  resulting  architecture  converges 
fast,  produces  more  reliable  results  and 
reduces  the  computational  burden 
considerably  when  compared  to  fully 
connected  neural  networks. 

1  Introduction 

The  success  of  solving  the  problems 
using  neural  networks  is  primarily  dependent 
upon  the  architecture  of  the  neural  network  and 


learning  strategy  used  to  train  the  neural 
networks.  The  functioning  of  the  biological 
neuron  is  known,  but  the  functioning  of  the 
network  of  biological  neuron  is  still  remains  as 
a  mystery.  The  basic  aim  of  the  artificial  neural 
network  research  is  to  reveal  the  functioning 
networks  of  neuron  in  the  human  brain.  The 
number  of  the  input  units  and  the  number  of 
output  units  of  a  neural  network  for  solving  a 
given  problem  are  chosen  according  to  the  size 
of  the  preprocessed  input  and  target  output 
pattern  respectively.  The  performance  of  a 
neural  network  is  solely  dependent  upon  the 
nature  processing  take  place  in  the  hidden 
layers.  We  proposed  a  Novel  Paradigm  of 
Neural  Network  Architecture,  which  has  new 
type  of  the  hidden  layer  based  on  physiological 
properties  of  biological  neuron. 

2  Biological  Background 

Nerve  cells  are  encased  in  a  semi- 
permeable  membrane  that  permits  some 
substances  to  pass  through  the  membrane, 
while  others  are  kept  out.  As  the  consequence, 
there  are  more  number  negative  ions  are  inside 
than  outside  the  cell  membrane  resulting  in  a 
membrane  potential.  This  membrane  potential 
is  called  the  resting  potential  of  the  cell.  A  cell 
in  the  resting  state  is  said  to  be  polarized. 
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When  a  section  of  the  cell  membrane  is 
excited  by  the  flow  of  ionic  current  or  by  some 
form  of  externally  applied  energy.  The 
membrane  changes  its  characteristics  and 
begins  to  allow  some  of  the  sodium  ions  to 
enter  this  movement  of  sodium  ions  into  the 
cell.  As  a  result,  the  cell  has  a  slightly  positive 
potential  on  inside  due  to  the  imbalance  of 
potassium  ions.  This  potential  is  known  as  the 
action  potential.  A  cell  that  has  been  excited 
and  that  displays  an  action  potential  is  said  to 
be  depolarized;  the  process  of  changing  from 
the  resting  state  to  the  action  potential  is  called 
depolarization. 

Once  the  rush  of  sodium  ions  through 
the  cell  membrane  has  stopped,  the  ionic 
current  that  lowered  the  barrier  to  sodium  ions 
are  no  longer  present  and  the  membrane  reverts 
back  to  its  original,  selectively  permeable 
condition,  where  in  the  passage  of  sodium  ions 
from  the  outside  to  the  inside  of  the  cell  is 
again  blocked.  A  resting  potential  is  restored 
back  after  a  long  time,  this  is  done  by  an  active 
process  called  sodium  potassium  pump.  The 
process  of  changing  the  depolarized  to 
polarized  state  is  called  repolarization. 

Regardless  of  the  method  by  which  a 
cell  is  excited  or  the  intensity  of  stimulus,  the 
action  potential  is  always  the  same  for  any 
given  cell.  This  is  known  as  the  all  or  nothing 
law.  Following  the  generation  of  an  action 
potential,  there  is  a  brief  period  of  time  during 
which  the  cell  can  not  respond  to  any  new 
stimulus.  This  period  is  called  the  absolute 
refractory  period  and  last  about  1  msec,  in 
nerve  cells.  A  relative  refractory  period  occurs 
following  the  absolute  refractory  period,  during 
this  relative  refractory  period  another  action 
potential  can  be  triggered  by  much  stronger 
stimulation.  In  nerve  cells,  the  relative 
refractory  period  lasts  several  milliseconds.  [ 
Lesile  Cromwell  et  ah,  1980  ] 

2.1  Propagation  of  Action  Potential 

When  a  cell  is  excited  and  generates  an 
action  potential  ionic  currents  begin  to  flow. 
This  process  can,  in  turn,  excite  neighbouring 


cells  or  adjacent  areas  of  the  same  cell.  In 
nature,  nerve  cells  are  excited  only  near  their 
input  end.  As  the  action  potential  travels  down 
the  fiber,  it  can  not  re-excite  the  portion  of  the 
fiber  immediately  upstream,  because  of  the 
refractory  period  that  follows  the  action 
potential.  The  rate  of  which  an  action  potential 
moves  down  a  fiber  is  called  the  propagation 
rate.  In  nerve  fibers  the  propagation  rate  is  is 
from  20  to  140  meters  per  second. 

3  Motivation 

The  physiology  of  nerve  cells  indicate 
that  there  are  three  states  for  neuron  namely 

I.  Resting  State, 

II.  Excitory  State  and 
in.  Inhibitory  State. 

One  option  is  the  design  of  neural 
network  architecture  based  on  the  above 
information  by  assigning  three  different  values 
to  each  of  the  three  different  states  of  a 
processing  unit.  This  tri-logic  architecture  ends 
with  huge  calculation  during  the  training  phase 
of  the  neural  network.  The  results  obtained  as 
well  as  the  performance  of  the  neural  network 
are  not  very  satisfactory. 

This  drawback  made  us  to  think  about 
alternative  formalism  to  implement  the  three 
different  states  for  neural  network  units.  In  our 
proposed  new  neural  network  architecture  we 
maintain  the  binary  states  for  each  unit  in  the 
neural  network  with  values  0  and  1 .  But  1  state 
represents  the  excitory  state  which  happen 
when  weighted  sum  of  all  inputs  from  1  state 
units  of  the  immediately  lower  layer  to  a 
particular  unit  exceeds  the  threshold  value 
determined  by  an  activation  function  and  the  0 
state  represents  either  resting  state  or  inhibitory 
states.  The  inhibitory  and  not  the  resting  state 
contribute  to  neural  network  processing.  Our 
proposed  novel  neural  network  architecture 
incorporate  inhibitory  state  mechanism. 

4  Dual  Nature  Hidden  Layer  Neural 
Networks 


Our  new  experimental  model  employs 
three  layers  topology  and  this  technique  can  be 
extended  to  multiple  hidden  layers.  The  bottom 
layer  of  units  is  the  input  layer  -  the  only  units 
in  the  network  that  receive  external  input.  The 
top  layer  is  the  output  layer.  The  middle  layer 
is  divided  into  two  parts.  The  first  part  consist 
of  units  that  are  responsible  for  excitation  units 
in  the  output  layer  i.e.  the  1  state  and  the 
second  part  consist  of  units  that  are  responsible 
for  inhibition  of  the  units  in  the  output  layer  i.e. 
the  0  states.  These  three  layers  are 
interconnected  as  shown  in  the  fig.  1.  Each 
processing  unit  in  the  input  layer  is  connected 
separately  to  every  processing  unit  in  the 


inhibitory  part  and  excitory  part  of  the  hidden 
layer.  Similarly  every  units  in  the  inhibitory 
part  and  excitory  part  of  the  hidden  layer  are 
connected  to  all  the  units  in  the  output  layer. 
The  units  are  not  connected  to  other  units  in  the 
same  layer  The  straight  line  links  indicates  the 
connections  responsible  for  producing  the  1 
state  in  the  output  layer  and  dotted  line  links 
indicates  the  connection  responsible  for 
producing  the  0  state  in  the  output  layer.  A 
significant  change  in  this  model  is  that  the 
separate  processing  unit  for  producing  one  and 
zero  states  of  the  output  layer  units. 


is  excitory 


is  inhibitory 


Fig.l  :  The  Dual  Nature  Hidden  Layer  Neural  Networks. 


5  Results 

We  constructed  a  few  examples  like 
ECG  signal  pattern  classification  etc  and  tested 
them  with  the  dual  nature  hidden  layer  neural 
network  against  the  fully  connected  back 
propagation  three  layer  network.  Our 
architecture  proved  to  be  superior  than  back 
propagation  in  all  these  cases. 

Following  salient  features  are  observed: 

o  The  number  of  interconnections  are 
greatly  reduced. 

o  The  computational  burden  is 
considerably  less  and  hence  processing 
is  faster. 

o  The  unique  special  feature  of  this 
architecture  is  ‘zeros’  and  ‘ones’  in  the 
output  layer  are  processed  separately, 
o  Faster  learning  and  the  reliability  of 
producing  the  correct  results  are  very 
high. 


7  Conclusions  and  discussion 

In  this  paper,  we  have  presented  a 
scheme  called  dual  nature  hidden  layer  neural 
network  which  incorporate  the  inhibitory 
mechanism  of  the  biological  neuron.  This 


architecture  produce  more  reliable  output 
which  can  used  in  medical  pattern  classification 
problems.  It  converges  fast  and  show  a  much 
superior  results  than  the  three  layer  back 
propagation  neural  network.  This  network 
yields  best  results  for  input  without  having 
noise.  For  noisy  inputs  suitable  digital  filters 
can  be  used  to  obtain  the  best  results.  The 
relative  proportion  excitory  and  inhibitory  unit 
can  be  determined  by  calculating  the  number 
of  ones  and  zeros  in  all  the  target  patterns  for 
obtaining  the  best  results.  The  total  number  of 
interconnection  are  greatly  reduced  due  to 
division  in  the  hidden  layer.  This  results  in 
faster  processing. 
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